Parallels between sensory and motor information processing

The computational problems solved by the sensory and motor systems appear very different: one has to do with inferring the state of the world given sensory data, the other with generating motor commands appropriate for given task goals. However recent mathematical developments summarized in this chapter show that these two problems are in many ways related. Therefore information processing in the sensory and motor systems may be more similar than previously thought – not only in terms of computations but also in terms of algorithms and neural representations. Here we explore these similarities as well as clarify some differences between the two systems. Similarity between inference and control: an intuitive introduction Consider a control problem where we want to achieve a certain goal at some point in time in the future – say, grasp a coffee cup within 1 sec. To achieve this goal, the motor system has to generate a sequence of muscle activations which result in joint torques which act on the musculo-skeletal plant in such a way that the fingers end up curled around the cup. Actually the motor system does not have to compute the entire sequence of muscle activations in advance. All it has to compute are the muscle activations right now, given the current state of the world (including the body) and some description of what the goal is. If the system is capable of performing this computation, then it will generate the resulting muscle activations, the clock will advance to the next point in time, and the computation will be repeated. How can this control problem be interpreted as an inference problem? Instead of aiming for a goal in the future, imagine that the future is now and the goal has been achieved. More precisely, shift the time axis by 1 sec and create a fictive sensory measurement corresponding to the hand grasping the cup. The inference problem is now as follows: given that the fingers are around the cup and that the world was at a certain state 1 sec ago, infer the muscle activations which caused the observed state transition. As in the control problem, all that needs to be inferred are the muscle activations at a single point in time (1 sec ago); if this can be done then the clock will advance (to say 0.99 sec ago) and the computation will be repeated.